from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.make_report()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following parameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score_sklearn	accuracy_score_onnx	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.014	0.001	NaN	5.631	0.0	-1	1	NaN	0.663	19.228	0.032	0.001	0.001	See	See
3	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.014	0.001	NaN	5.613	0.0	-1	5	NaN	1.000	0.365	0.013	0.039	0.039	See	See
6	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.014	0.001	NaN	5.692	0.0	1	100	NaN	0.882	19.487	0.051	0.001	0.001	See	See
9	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.014	0.001	NaN	5.738	0.0	-1	100	NaN	1.000	0.382	0.010	0.037	0.037	See	See
12	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.014	0.001	NaN	5.576	0.0	1	5	NaN	0.896	4.578	0.047	0.003	0.003	See	See
15	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.015	0.001	NaN	5.267	0.0	1	1	NaN	1.000	0.292	0.009	0.052	0.052	See	See
18	KNeighborsClassifier_brute_force	fit	100000	100000	2	0.005	0.000	NaN	0.330	0.0	-1	1	NaN	0.929	4.567	0.093	0.001	0.001	See	See
21	KNeighborsClassifier_brute_force	fit	100000	100000	2	0.005	0.000	NaN	0.300	0.0	-1	5	NaN	1.000	0.286	0.007	0.019	0.019	See	See

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score_sklearn	accuracy_score_onnx	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.287	0.096	NaN	0.0	0.002	-1	1	0.663	1.000	0.369	0.008	6.192	6.193	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.027	0.003	NaN	0.0	0.027	-1	1	1.000	0.757	19.867	0.079	0.001	0.001	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	3.151	0.049	NaN	0.0	0.003	-1	5	0.757	0.882	19.416	0.052	0.162	0.162	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1	100	0.029	0.002	NaN	0.0	0.029	-1	5	1.000	1.000	0.372	0.006	0.077	0.077	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.536	0.050	NaN	0.0	0.003	1	100	0.882	1.000	0.376	0.010	6.741	6.744	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1	100	0.028	0.004	NaN	0.0	0.028	1	100	1.000	0.757	19.327	0.020	0.001	0.001	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1000	100	3.182	0.056	NaN	0.0	0.003	-1	100	0.882	0.663	19.729	0.020	0.161	0.161	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.028	0.003	NaN	0.0	0.028	-1	100	1.000	1.000	0.374	0.007	0.074	0.074	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.518	0.027	NaN	0.0	0.003	1	5	0.757	1.000	0.294	0.009	8.567	8.571	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.026	0.001	NaN	0.0	0.026	1	5	1.000	0.922	4.445	0.066	0.006	0.006	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	1.626	0.013	NaN	0.0	0.002	1	1	0.663	0.929	4.528	0.059	0.359	0.359	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1	100	0.025	0.001	NaN	0.0	0.025	1	1	1.000	1.000	0.289	0.008	0.086	0.087	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1000	2	1.839	0.051	NaN	0.0	0.002	-1	1	0.896	1.000	0.296	0.018	6.221	6.232	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1	2	0.007	0.002	NaN	0.0	0.007	-1	1	1.000	0.922	4.497	0.017	0.002	0.002	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1000	2	2.782	0.028	NaN	0.0	0.003	-1	5	0.922	0.896	4.413	0.048	0.631	0.631	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	2	0.009	0.005	NaN	0.0	0.009	-1	5	1.000	1.000	0.282	0.005	0.032	0.032	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following parameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score_sklearn	accuracy_score_onnx	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.087	0.059	NaN	0.026	0.0	-1	1	NaN	0.929	151.324	0.000	0.020	0.020	See	See
3	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.938	0.060	NaN	0.020	0.0	-1	5	NaN	1.000	3.051	0.308	1.291	1.297	See	See
6	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.881	0.107	NaN	0.021	0.0	1	100	NaN	0.951	143.419	0.000	0.027	0.027	See	See
9	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.896	0.059	NaN	0.021	0.0	-1	100	NaN	1.000	3.088	0.316	1.262	1.268	See	See
12	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.845	0.070	NaN	0.021	0.0	1	5	NaN	0.891	0.047	0.014	81.048	84.671	See	See
15	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.899	0.123	NaN	0.021	0.0	1	1	NaN	1.000	0.006	0.001	662.419	666.302	See	See
18	KNeighborsClassifier_kd_tree	fit	1000	1000	2	0.001	0.001	NaN	0.015	0.0	-1	1	NaN	0.894	0.067	0.001	0.016	0.016	See	See
21	KNeighborsClassifier_kd_tree	fit	1000	1000	2	0.001	0.000	NaN	0.023	0.0	-1	5	NaN	1.000	0.006	0.001	0.120	0.121	See	See

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score_sklearn	accuracy_score_onnx	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	0.987	1.294	NaN	0.0	0.001	-1	1	0.929	1.000	3.170	0.362	0.311	0.314	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.003	0.001	NaN	0.0	0.003	-1	1	1.000	0.946	147.779	0.000	0.000	0.000	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	1.147	0.474	NaN	0.0	0.001	-1	5	0.946	0.951	143.112	0.000	0.008	0.008	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.004	0.001	NaN	0.0	0.004	-1	5	1.000	1.000	3.044	0.247	0.001	0.001	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	5.905	0.750	NaN	0.0	0.006	1	100	0.951	1.000	3.067	0.266	1.925	1.933	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.003	0.001	NaN	0.0	0.003	1	100	1.000	0.946	144.037	0.000	0.000	0.000	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	3.249	0.417	NaN	0.0	0.003	-1	100	0.951	0.929	147.300	0.000	0.022	0.022	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.005	0.001	NaN	0.0	0.005	-1	100	1.000	1.000	3.028	0.291	0.002	0.002	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	1.827	0.469	NaN	0.0	0.002	1	5	0.946	1.000	0.007	0.001	266.041	270.567	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.002	0.000	NaN	0.0	0.002	1	5	1.000	0.911	0.043	0.001	0.036	0.036	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	1.016	0.402	NaN	0.0	0.001	1	1	0.929	0.894	0.065	0.002	15.670	15.677	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.001	0.000	NaN	0.0	0.001	1	1	1.000	1.000	0.006	0.000	0.201	0.202	See	See
19	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.016	NaN	0.0	0.000	-1	1	0.891	1.000	0.006	0.001	6.572	6.698	See	See
20	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.003	0.000	NaN	0.0	0.003	-1	1	1.000	0.911	0.045	0.002	0.062	0.062	See	See
22	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.035	0.004	NaN	0.0	0.000	-1	5	0.911	0.891	0.060	0.032	0.589	0.666	See	See
23	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.003	0.000	NaN	0.0	0.003	-1	5	1.000	1.000	0.006	0.000	0.437	0.438	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following parameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	accuracy_score_sklearn	accuracy_score_onnx	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	fit	100000	100000	100	121.509	0.0	300	0.001	0.001	NaN	0.824	0.591	0.032	205.693	205.997	See	See

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	accuracy_score_sklearn	accuracy_score_onnx	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.191	0.004	300	0.004	0.0	0.824	1.0	0.476	0.016	0.401	0.402	See	See